Dependency hell is a well-known pain point in the development of large software projects and machine learning (ML) code bases are not immune from it. In fact, ML applications suffer from an additional form, namely, "data source dependency hell". This term refers to the central role played by data and its unique quirks that often lead to unexpected failures of ML models which cannot be explained by code changes. In this paper, we present an automated dependency mapping framework that allows MLOps engineers to monitor the whole dependency map of their models in a fast paced engineering environment and thus mitigate ahead of time the consequences of any data source changes (e.g., re-train model, ignore data, set default data etc.). Our system is based on a unified and generic approach, employing techniques from static analysis, from which data sources can be identified reliably for any type of dependency on a wide range of source languages and artefacts. The dependency mapping framework is exposed as a REST web API where the only input is the path to the Git repository hosting the code base. Currently used by MLOps engineers at Microsoft, we expect such dependency map APIs to be adopted more widely by MLOps engineers in the future.

Training effective embodied AI agents often involves manual reward engineering, expert imitation, specialized components such as maps, or leveraging additional sensors for depth and localization. Another approach is to use neural architectures alongside self-supervised objectives which encourage better representation learning. In practice, there are few guarantees that these self-supervised objectives encode task-relevant information. We propose the Scene Graph Contrastive (SGC) loss, which uses scene graphs as general-purpose, training-only, supervisory signals. The SGC loss does away with explicit graph decoding and instead uses contrastive learning to align an agent's representation with a rich graphical encoding of its environment. The SGC loss is generally applicable, simple to implement, and encourages representations that encode objects' semantics, relationships, and history. Using the SGC loss, we attain significant gains on three embodied tasks: Object Navigation, Multi-Object Navigation, and Arm Point Navigation. Finally, we present studies and analyses which demonstrate the ability of our trained representation to encode semantic cues about the environment.

We present DyFOS, an active perception method that Dynamically Finds Optimal States to minimize localization error while avoiding obstacles and occlusions. We consider the scenario where a ground target without any exteroceptive sensors must rely on an aerial observer for pose and uncertainty estimates to localize itself along an obstacle-filled path. The observer uses a downward-facing camera to estimate the target's pose and uncertainty. However, the pose uncertainty is a function of the states of the observer, target, and surrounding environment. To find an optimal state that minimizes the target's localization uncertainty, DyFOS uses a localization error prediction pipeline in an optimization search. Given the states mentioned above, the pipeline predicts the target's localization uncertainty with the help of a trained, complex state-dependent sensor measurement model (which is a probabilistic neural network in our case). Our pipeline also predicts target occlusion and obstacle collision to remove undesirable observer states. The output of the optimization search is an optimal observer state that minimizes target localization uncertainty while avoiding occlusion and collision. We evaluate the proposed method using numerical and simulated (Gazebo) experiments. Our results show that DyFOS is almost 100x faster than yet as good as brute force. Furthermore, DyFOS yielded lower localization errors than random and heuristic searches.

Green Security Games with real-time information (GSG-I) add the real-time information about the agents' movement to the typical GSG formulation. Prior works on GSG-I have used deep reinforcement learning (DRL) to learn the best policy for the agent in such an environment without any need to store the huge number of state representations for GSG-I. However, the decision-making process of DRL methods is largely opaque, which results in a lack of trust in their predictions. To tackle this issue, we present an interpretable DRL method for GSG-I that generates visualization to explain the decisions taken by the DRL algorithm. We also show that this approach performs better and works well with a simpler training regimen compared to the existing method.

多机器人覆盖计划问题的集中式方法缺乏可扩展性。基于学习的分布式算法除了将面向数据的功能生成功能带入表格外，还提供了可扩展的途径，从而允许与其他基于学习的方法集成。为此，我们提出了一个基于学习的，可区分的分布式覆盖范围计划（D2COPL A N），该计划者与专家算法相比在运行时和代理数量上有效地扩展，并与经典分布式算法相同。此外，我们表明D2Coplan可以与其他学习方法无缝地结合到端到端的学习方法，从而提供了比单独训练的模块更好的解决方案，从而打开了进一步的研究，以进一步研究以经典方法难以捉摸的任务。

我们研究了合作航空航天车辆路线应用程序的资源分配问题，其中多个无人驾驶汽车（UAV）电池容量有限和多个无人接地车辆（UGV），这也可以充当移动充电站，需要共同实现诸如持续监视一组要点之类的任务。由于无人机的电池能力有限，他们有时必须偏离任务才能与UGV进行集合并得到充电。每个UGV一次可以一次提供有限数量的无人机。与确定性多机器人计划的先前工作相反，我们考虑了无人机能源消耗的随机性所带来的挑战。我们有兴趣找到无人机的最佳充电时间表，从而最大程度地减少了旅行成本，并且在计划范围内没有任何无人机在计划范围内取消收费的可能性大于用户定义的公差。我们将此问题（{风险意识召集集合问题（RRRP））}作为整数线性程序（ILP），其中匹配的约束捕获资源可用性约束，而背包约束捕获了成功概率约束。我们提出了一种求解RRRP的双晶格近似算法。在一个持续监测任务的背景下，我们证明了我们的制定和算法的有效性。

在许多机器人应用中，要执行已知，刚体对象及其随后的抓握的6多-DOF姿势估计的环境设置几乎保持不变，甚至可能是机器人事先知道的。在本文中，我们将此问题称为特定实例的姿势估计：只有在有限的一组熟悉的情况下，该机器人将以高度准确性估算姿势。场景中的微小变化，包括照明条件和背景外观的变化，是可以接受的，但没有预期的改变。为此，我们提出了一种方法，可以快速训练和部署管道，以估算单个RGB图像的对象的连续6-DOF姿势。关键的想法是利用已知的相机姿势和刚性的身体几何形状部分自动化大型标记数据集的生成。然后，数据集以及足够的域随机化来监督深度神经网络的培训，以预测语义关键。在实验上，我们证明了我们提出的方法的便利性和有效性，以准确估计物体姿势，仅需要少量的手动注释才能进行训练。

基于腿部机器人的基于深的加固学习（RL）控制器表现出令人印象深刻的鲁棒性，可在不同的环境中为多个机器人平台行走。为了在现实世界中启用RL策略为类人类机器人应用，至关重要的是，建立一个可以在2D和3D地形上实现任何方向行走的系统，并由用户命令控制。在本文中，我们通过学习遵循给定步骤序列的政策来解决这个问题。该政策在一组程序生成的步骤序列（也称为脚步计划）的帮助下进行培训。我们表明，仅将即将到来的2个步骤喂入政策就足以实现全向步行，安装到位，站立和攀登楼梯。我们的方法采用课程学习对地形的复杂性，并规避了参考运动或预训练的权重的需求。我们证明了我们提出的方法在Mujoco仿真环境中学习2个新机器人平台的RL策略-HRP5P和JVRC -1-。可以在线获得培训和评估的代码。

本文提出了基于联邦学习（FL）的SMAR T医疗保健系统，其中医疗中心（MCS）使用患者收集的数据训练本地模型，并将模型权重以基于区块链的强大框架将原始数据发送给矿工，而无需共享原始数据隐私保护进行审议。我们通过最大化效用并最大程度地降低了MCS在基于区块链的框架为基础的分布式医疗保健数据上学习有效模型的损失功能来提出优化问题。我们在两个阶段提出了一个解决方案：首先，提供一种稳定的基于匹配的关联算法，以最大程度地提高矿工和MC的实用性，然后使用随机梯度下降（SGD）算法解决损失最小化，该算法在差异隐私（DP）和区块链下使用FL技术。此外，我们合并了区块链技术，以在拟议的基于FL的框架中提供抗性和分散的模型重量共享。通过模拟现实世界中的医疗保健数据比较其他最先进的技术，该模型的有效性显示了。

在本文中，我们提出了一种新颖的重尾随机策略梯度（HT-PSG）算法，以应对连续控制问题中稀疏奖励的挑战。稀疏的奖励在连续控制机器人技术任务（例如操纵和导航）中很常见，并且由于对状态空间的价值功能的非平凡估计而使学习问题变得困难。这需要奖励成型或针对稀疏奖励环境的专家演示。但是，获得高质量的演示非常昂贵，有时甚至是不可能的。我们提出了一个重型策略参数化，以及基于动量的策略梯度跟踪方案（HT-SPG），以引起对算法的稳定探索行为。提出的算法不需要访问专家演示。我们测试了HT-SPG在连续控制的各种基准测试任务上的性能，并具有稀疏的奖励，例如1d Mario，病理山车，Openai体育馆的稀疏摆和稀疏的Mujoco环境（Hopper-V2）。就高平均累积奖励而言，我们在所有任务中表现出一致的性能提高。 HT-SPG还证明了最低样品的收敛速度提高，从而强调了我们提出的算法的样品效率。